Optimize unlimited broadcast performance#623
Merged
Merged
Conversation
5847a64 to
946c13a
Compare
SaveBroadcastMessage (called when a client is about to receive a broadcast message) tends to fail due to context cancelation (timeout) in large broadcasts. That's because broadcast messages to clients are processed sequentially using a mutex to prevent parallel writes to the task's broadcast allocation, which would lead to various issues depending on the db implementation. *Context: An allocation is a temporary, local quota of the total broadcast limit assigned to a specific task. On startup, tasks create an allocation of an active broadcast (usually 20% of the remaining messages), which allows it to send broadcast messages to clients without syncing with other tasks on how many messages are still remaining.* For broadcasts with no limit (`db.BroadcastUnlimited`), tasks generally don't need to care how many messages others have sent, and we can therefore skip the allocation mechanism and process messages in parallel. With this change: - Unlimited broadcasts (`Limit == db.BroadcastUnlimited`) skip database allocation creation. - Senders call `SaveBroadcastMessage` with an empty `AllocationID` for unlimited broadcasts. - Datastores (SQLite, MySQL, Spanner) skip updating `broadcast_allocations` if the `AllocationID` is empty. - The `Manager` skips acquiring the in-memory `i.lock` for unlimited broadcasts, allowing concurrent processing. - The `sent` count is tracked in-memory and flushed to `broadcasts.sent` when the manager is closed or the broadcast is cleaned up. PiperOrigin-RevId: 942179136
946c13a to
b33bdde
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Optimize unlimited broadcast performance
SaveBroadcastMessage (called when a client is about to receive a broadcast message) tends to fail due to context cancelation (timeout) in large broadcasts. That's because broadcast messages to clients are processed sequentially using a mutex to prevent parallel writes to the task's broadcast allocation, which would lead to various issues depending on the db implementation.
Context: An allocation is a temporary, local quota of the total broadcast limit assigned to a specific task. On startup, tasks create an allocation of an active broadcast (usually 20% of the remaining messages), which allows it to send broadcast messages to clients without syncing with other tasks on how many messages are still remaining.
For broadcasts with no limit (
db.BroadcastUnlimited), tasks generally don't need to care how many messages others have sent, and we can therefore skip the allocation mechanism and process messages in parallel.With this change:
Limit == db.BroadcastUnlimited) skip database allocation creation.SaveBroadcastMessagewith an emptyAllocationIDfor unlimited broadcasts.broadcast_allocationsif theAllocationIDis empty.Managerskips acquiring the in-memoryi.lockfor unlimited broadcasts, allowing concurrent processing.sentcount is tracked in-memory and flushed tobroadcasts.sentwhen the manager is closed or the broadcast is cleaned up.